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ABSTRACT 



This paper Illustrates an Important Interplay between laboratory 
research on feedback and development of Instructional materials. Ex* 
ploratory work suggests that the products of behavior or ''product 
feedback*** are functionally different from differential reinforcement. 
Product feedback provides the means by which self '-^correction can take 
place; differential reinforcement does not. This basic research leads 
to a particular analysis of feedback in two types of curricula: A 
writing curriculum that makes use of pencil and paper materials and 
a spelling curriculum that is part of a larger system of computer 
assisted instruction. The basic research and instructional implica-- 
tions suggest a need for a program of research on feedback and under--* 
scoreSthe importance of discrimination training as a precursor to 
any skilled performance. 



"Produce Feedback" 
Pamela iteadowcrofc 



Introduction 

As yoa have heard from Ms. Doran, operant research on errorless 
learning grew out of the early applications of behavioral principles 
in programmed instruction. Low error rates accompanied successful 
learning of verbal material. Yet the prior theory had dictated the 
necessity of errors for learning. Terrace (1963) brought this dis- 
crepancy into the lab and established an elegant program of research 
on errorless learning which continues to affect instructional prac- 
tices and laboratory research. 

Ue now turn to a current gap in our science of behavior. The 
acquisition and maintenance of behavior by reinforcement is amply 
supported both in the lab and in the natural environment. It is a 
well knovm principle that responses which are followed by positive 
consequences are strengthened. Even though contingent reinforcement 
is response produced, little consideration has been given to the 
products of behavior, such as a spoken word, a sung note, and a 
printed letter. These products serve as cues for subsequent re- 
sponding. In this sense the products of behavior are feedback- 
response produced stimuli that determine the form of future behavior. 

In the following talk I will present an interpretative behav- 
ioral analysis of produce feedback, how this analysis inspired some 
laboratory research we are currently exploring and the Implications 
for thts experimental analysis in instructional practices. 
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A Behavioral Analysis 

Consider an infant learning to talk. Nortoally skilled speech 
develops by learning to match self-produced sounds to speech sounds. 
Discriminations between self-produced sounds and appropriate speech 
sounds are initially established by the differential reactions of the 
child's verbal cotamunity. At first any approximation to a speech 
sound meets with parental joy while as these early sounds are es- 
tablished the joy criterion shifts more and more towards correct 
speech forms. This training involves both the differentiation of 
the child's own vocal-muscle patterns as well as learning to hear 
the differences among the sounds these responses produce. In the 
processes of vocal*-muscle differentiation and sound discrimination 
the child learns the relevant response and stimulus dimensions for 
his or her particular language. 

Eventually reinforcement for correct speech production shifts 
from the external reinforcing community to more immediate, self- 
generated sources. Tliis shift makes continued shaping and main- 
tenance of good speech automatic. That is, when the child's "ear" 
has been trained to discriminate correct speech sounds, he or she 
can immediately judge the adequacy of self-produced sound as well 
as correct inappropriate utterances. 

Discriminating or judging the adequacy of a self-produced 
sound determines subsequent behavior. If the sound is judged as 
adequate, then reinforcement is automatic — no external confirmation 
is necessary. If the self-produced sound is judged as inadequate, 

O 
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then a shift In producing a sound closer to the speech sound Is 
Initiated. This shift or self-correction is possible because of 
the child ^s earlier experience in which an incorrect sound came to 
serve as a cue for changing the response along the relevant dimen** 
sions for chat language. The magnitude and direction of change 
depend on the previously trained sound discrimination. That is, 
only when feedback sounds have been discriminated can the learner 
self-correct. 

The Implications of discriminated feedback stretch across all 
productive behavior. Even attempts at "remembering** require an 
analysis in terms of discriminated feedback* We have all had a 
tip-of- the- tongue experience where an answer or name is almost 
recalled but not quite. The following scenario borrowed from Adams 
(1967) would not be uncommon: 

As Che subject runs through a series of similar responses 
he has the clear feeling chat he knows the correct response, 
has it on the tip of his tongue, and can give it in Just a 
moment. Moreover, the subject not only knows he Is in 
error and must reject wrong responses, buc also knows Che 
relacive magnicude of che errors. For example, a subjecc 
night be asked Co give che caplcal of che scace of Illinois* 
He mighc say, **The caplcal of Illinois is Bloomingcon. No 
thac*s wrong! Planfield! No, chac isn^c ic eicher! 
Summerfleld! Thac's close! Springfield! Thac's ic!" 
Our subJecC certainly seems to know wtiat his first 
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response is a relaclvely large error and should summarily 
be rejected, chat the second response has smaller error 
and is In che general region of che correct response, that 
the third response Is very close to being correct and so 
on. (Adams, 1967, pp. 284-285) 
Here the Indlvdual tries to recall Springfield. After each approX'^ 
imatlon the guess Is automatically corrected with a shift towards a 
city that sounds more and more like Springfield. 

But what guides this sustained guessing? The currect situation, 
*Vhat is the capital of Illinois?'* is similar to past situations in 
which the response ''Springfield'* was externally reinforced by a 
teacher or by textual material. However, under conditions where 
reinforcement for "Springfield" is Infrequent we can expect other 
responses that share stimulus elements with "Springfield" to have 
high probabilities of occurrence. Each of these high probability 
responses — Planfleld, Bloomlngton» Summerfleld — by the speaker are 
evaluated by the listener, both of whom in this Instance are within 
the same skin. The degree to which the listener discriminates the 
appropriateness of the guess will determine when guessing stops. 
The shift of each guess along the "Springfield continuum," takes 
place because each inappropriate guess serves as a self*prompt for 
strengthening the next guess. Such automatic shaping by the speaker-* 
listener would not be possible without discriminated feedback and 
feedback (or self-produced products) will not be discrixnLnated with- 
out previous exposure to contingencies in which a response is made 
and is differentially reinforced. 
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Experimental Analysis of Feedback 

In both Che speech and memory examples feedback Is in che form 
of a produce produced by Che Individual* Produces of action Indicate 
the degree Co which behavior Is adequate and consequently can In- 
fluence the form of future performance. Such a formulation of feed-* 
back Is descriptively different from simple differential relnforcemant 
in which the correct response is reinforced by some arbitrary stimulus 
and the Incorrect response Is not reinforced. 

Since our interpretation of feedback differs from traditional 
behavioral analyses we went to the lab to analyze the effects of 
self-produced products. The pilot work I will describe and the early 
data from it suggest that feedback enhances acquisition of productive 
behavior over and abo\c the effects of differential reinforcement. 

Insert Slide 1 about here 

Children were trained to respond to five locations on a long response 
key in the presence of five progressively flatter ellipses. Discrim- 
inations among these ellipses had been previously established in a 
same-different task. In the upper half of this slide the figures 
illustrate the same-different task in which each child was presented 
with an ellipse on the center key and a comparison ellipse on the 
right hand key. They were then told to touch the right hand key if 
the two pictures were the same or the blank, left hand key if the 
figures were not alike. In the bottom part of the slide are figures 
thjit Illustrate the training procedures for learning the five elllpse- 
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position correspondences* An ellipse was presented in the upper- 
center key and the children had to find the correct place for this 
picture along the bottom long key. Touching any of the five areas 
produced differential reinforcement* A correct response produced 
a chime and mto^s, whereas pushing the wrong area produced an inter"* 
trial-interval. Some children » In addition to this differential 
reinforcement, received productive feedback in the form of the 
ellipse appropriate to the touched area* Learning the correspondences 
between ellipses and areas when followed by arbitrary reinforcement 
(chimes and m&m^s) was slow; whereas when each response produced an 
ellipse that was proportional to that response's position along the 
long key, learning was quite rapid. 

On this data slide you see the individual results for lean ing 
the ellipse-position correspondences for positions 2 and 4* Each 
step-up indicates an error on that trial. In the top part of the 
slide these no-feedback data show many errors; in fact, the first 
four children never reached the criterion of eight out of ten con- 
secutively correct trials. In contrast, the bottom half of the 
slide shows all children, whose responses pro'duced ellipse feedback, 
reached criterion with overall fewer errors. Therefore, only when 

Insert Slide 2 about here 

the ellipse-to-position relation indicated the degree of correctness 
of the response was learning effective. 

Such results arc not predicted from operant work. In research 
on stimulus control responding in the presence of a particular stimulus 
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is succtssfully establlshtd when reinforcement Is differential. 
Typically, a discrete response, such as pressing one key, Is rein- 
forced during the experimental stimulus and not reinforced during 
other stimuli. This training is particularly rapid when reinforce- 
ment is differential and immediate. However, in the above experiment 
stimulus control by each of the ellipses is established More readily 
for the feedback condition even though every response in the other 
condition is differentially reinforced. 

We are continuing our look at productive behavior in the lab. 
This ongoing experimental analysis of feedback bears on operant 
theory. The evidence suggests some needed expansion of feedback 
effects. Also this research, inspired by behavioral descriptions 
of everyday behavior, feeds back into applied problems. It provides 
the empirical and conceptual base for the follo«^lng analyses of 
productive behavior in curriculum materials. 

Applied Uses of Feedback 

We chose two types of academic skills in different types of 
curriculum materials to analyze the uses of discriminated feedback. 
These Include printing, in paper and pencil format, and a spelling 
program from Computer Assisted Instruction (CAI). 

Printing. In most instructional approaches to teaching printing 
the child begins by copying or tracing over model letters (Enstrom & 
Trafford, 1966; >tonroe, 1973; Noble, 1974). Each finished product 
is then externally approved by the teacher. If the form is not good, 
the child has no means by which he/she can correct the form unless 
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additionml Infomatlon is glvtn conctrnlng how thm ton vis b«d» 
hov far from good it vast vhtthtr thm btginnlng or tnd strokti vtrt 
iQCorrtct, and so forth. Such dtmands for axttnsivt ttachtr tutorial 
timt Intvltably go unfulf Iliad. Small wondar printing is difficult 
to laarn and Judging from many adults ^ handwriting It s«oms that 
vtry ftw laarn good form. . 

A battar mathod to taach printing is ona that Insuraa discriminatad 
faadback or tha dlscriminacion of ona't own ptoducad lattar forma. 
If tha baglnning printar discrlminatas good form from bad, than ha/sha 
can Judga tha accuracy of thalr own lattars as tha lattar is baing 
producad. Diract discrimination training of good and bad lattar 
form should ba prominant in tha aarly parts of a writing curriculum 
so that cha baginnar doas not laarn bad form. 

Skinnar and Krakovar's (1968) program on cursiva writing shows 
axamplary usa of faadback for succassful writing. Ua would axpact 
to find good axamplas of dlscrlminacad faadback in thair program 
bacausa, in tha authors* own words: 

**Tha program strassas tha nacassity that aach child ba 
abla to discriminata batwatn badly formad and wall-formed 
cursiva letters. Contact with improperly formed letters 
Is a necessary and important part of learning to write. 
However I tha child is never required to form incorrect 
letters and so never has a chance to acquire bad hand** 
writing habits.'* 

It is an especially Interesting curriculum because the program 
uses paper and pencil materials and yet is able to insure responsiveness 



to ttudtnt bthavlor. It uMt a proettt ealltd wtltft-*«iKl*»tftft in 
vhieh ink from « tptelal ftlf pon tMett with tho ehMlc«lly trtatotf 
paptr to that llnot drava within appropriata boimda turn tha papar 
hroim> wharaaa inapproprlata, out-of'-bounda Imaa turn tha papar 
light yallow. Such aaaily dlacrlminatad, dlf farantial attoull 
ptovida iflOMidiata faadbaek for appropriata and Inapproprlata wvaa 
of tha pan. Aa tha child haglna to mva tha pan caralaaaly tha 
color of hia/har product providaa tha cuaa naadad for iwadiata 
correction. And aa tha progm progra^aaa tha **tolaranca** or araa 
in which tha child can Mva tha pan and atill produca brown di«iniahaa» 
Juat lika tha apaach axa^pla» tha critarion for good fom bacoMa 
a trie tar aa training procaada. 



Xnaart Slida 3 about hara 



Tha authora alao provida kay taaka for diactinination training 
of good fora. Group diacrimination taaka inclUda having tha taachar 
print on tha blackboard aavaral capital and lowar caaa lattara and 
than aak quaationa that amphaaiaa tha dimanaions of good and bad 
fora. In thia alida> tha top akaapla* tha taachar aaka quaationa 
about guida llnaa-- **Ooaa tha capital *A* touch tha top Una?** 
**Doaa tha *c* touch tha middla lina?** and ao forth. In tha mlddla 
axaapla tha taachar aakt tha childran what corractions ara raquirad 
to aaka these lattart better. Good font in both exaaplea follows 
guide line requireaents* In the bottom exaaple the teacher aaka 
questions about proper spacing to help establish needed discrimination 
for this aspect of good form. 
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In addition to these teacher directed activities, the program 
itself exposes the beginners to discrimination tasks. Here is a 



Insert Slide 4 about here 



sample of part of an exercise in which the beginning printers 
correct the poor letters. As in the whole program if their cor- 
rection marks are right , the marked paper turns brown; if wrong, 
the marked paper turns yellow. Here is another exercise sample 



Insert Slide 5 about here 



in which the printers trace*over the best letter. In both exercises 
the children discriminate good from bad forms. This experience, 
based on our experimental work and our interpretive analysis of pro** 
ductive feedback, would make self -correct ion of their own forms more 
possible. In the production exercises, seen in this slide, the dis- 

Insert Slide 6 about here 

criminating child is able to judge the accuracy of produced letters 
IS the supporting lines are faded out. 

Spelling 

Another set of instructional materials we analyzed in terms of 
discriminated feedback is a spelling curriculum developed at the 
Learning Research and Development Center at the University of 
Pittsburgh (Block, 1974; Simon and Simon, 1972). This curriculum 
is part of Computer Assisted Instruction and is called CAI spelling. 
(For a comprehensive beluvioral analysis of these curriculum ma- 
terials see Holland and Solomon, 1975). 
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Cofsputer assisted Instruction can. In theoryt 
readily meet the requirements for discriminated feedback. The 
sophisticated technology of the computer makes possible speedy 
discrimination training of the relevant dimensions for particular 
skilled production and can immediately provide the product results 
of subtle changes In behavior. 

However, provisions for discriminated feedback In computer 
assisted instruction are not as well developed as the technology. 
Often only a "yes" or "no" follows responses and though the appro- 
priate answer may be displayed for comparison with the student's 
product, the contingencies that establish the discriminative skills 
that are needed to change or maintain subsequent performance are, 
many times, lacking. 

The TRYSPL program from CAI spelling is a particularly inter- 
esting part of the curriculum because it is specifically designed 
to teach alternative graphemic spellings for a number of words and 
establish the discrimination of correct and incorrect spellings 
among these alternatives. We would expect, then, to find good uses 
of discriminated feedback. 

The authors of these materials base their instructional strategy 
on a "generate-and-test" phonetic process. Here the speller tries 
out different letters that correspond to the sounds in the word and 
when finished the word is recognized or discriminated as either 
correct or incorrect. If an attempted spelling is close enough to 
the correct one and is discriminated as such, this product, as in 
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the '•capital of Illinois" example, can serve as a prompt for an 
even closer or correct spelling. 

In the generatc-and-test process recognition of the spelled 
word requires previous experience with that word, or words with 
shared elements, in textual form. Without prior differential 
learning, the speller cannot discriminate the correctness of the 
generated spellings. It is no surprise that spelling errors are 
due to discrimination failures. 

Even though experience with the to-»be-spelled words is critical 
for choosing the correct alternative spelling, the TRYSPL program 
fails to build-in contingencies for word discrimination and thus 
falls to make efficient use of feedback. A sample of the protocol 
can be seen on the next slide. **Feedback'* as these authors use it 

Insert Slide 7 about here 

Include "Good'* :or a correct response and an appropriate message for 
an error such as "No," "No, look again," "No, the correct spelling 
is KNOCK*" No information is provided for any of the alternative 
spellings unless that spelling is chosen as correct. Then this 
choice is signaled as correct or not* The computer does eventually 
display the correct spelling which may, incidentally, help in es-- 
tabllshing discriminations of appropriate word spellings, but con- 
tingencies for explicitly training discriminations among the correct 
and Incorrect spellings are lacking in TRYSPL* We have seen that 
the alternative spellings can provide discriminated feedback for 
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correct spelling If the speller has previous relevant experience 
with the word. 

The authors of this curriculum did recognize the need for dls* 
crimination training of the spelling words. Consequently, the cur«- 
riculum was restricted to students who could already read all the 
spelling words. This target group would then have had necessary 
discrimination training in their previous reading activities. In 
addition, another program was created, called SPRUF, that gave 
direct discrimination training of the spelling words in the cur- 
riculum. 

Insert Slide 8 about here 

In this slide you see sample protocols for SPRUF. Students 
are given a sentence in which a word is left out such as **sour is 

to as lemon is to sugar." The appropriate word, "sweet," 

spelled in four alternative ways, is displayed below the sentence. 
The child chooses the word he/she recognizes as correct and if the 
choice is wrong the missed word is presented at the end of the list 
until all the words are correctly identified. Following this train- 
ing, the beginning speller would be able to discriminate the correct- 
ness of each of his/her spellings during the generate-and-test TRYSPL 
part of the curriculum. Just as the fellow was able to generate 
and reject city names until he hit upon Springfield, so too can the 
speller, with appropriate discrimination training, generate and test 
the accuracy of his/her own spellings. 
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Conclusion 

Our exploratory research and these curriculum analyses of dis- 
criminated feedback provide suggestions both to the curriculum de«* 
veloper and to the laboratory researcher* To the developer: Before 
a learner can skillfully execute productive behaviors, discrimination 
training among good and bad products must take place. Then the 
learner can judge the adequacy of their own products and self- 
correct. To the researcher: An open ended research program con- 
cerned with feedback properties exists. 
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PRETRAINING 



Prediscrimination 



Samplt Comparison 



■ [Q 



rtipdnd to this 
kty if fto match 



rttpond to this 
kty if match 



Nonprediscrimination 




rt fpSnd to this 
kay if no match 



rtipond to this 
kay if match 



TRAINING 



Correct Trial 



# 5 allipsa 





araa 5 with 
faadback 



Incorrect Trial 



# 5 allipsa 





»n» 3 with feedback 
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feedback 




Slide 2 
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Slldt 3 



Dl SCRIMINATION TRAINING 




INAPPROPRIATE FORM 



This is too little space. This 
is too much. Just riglit. 

SPACING 

If 

21 



Slide 4 
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The letters'with dots are wrong. Look 
at the first letter. Make the letters 
with dots look like the first letter. 

Make Bure the children com plete four letter t after each m odel. 
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• • 
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Slide 5 



Find and write the best letter. 



A 



Mako nura tho children undcmtund the directions. The ubcienco of u modd inuy ciiuno 
confusion. 



A 



I • 



: i 



,....4 



§••••• 



••••• 



I*' 



.V. 



ft* 
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•..4 
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• • t 
•••• • 



••V 
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Look at the finished letters on each h'ne. 
Moke the other letters look like these letters. 



Emphasize: 1. WritinK from loft to riKht 

2. Completion of each line before moving; on 




:a_x:zi:::::c:__ QIC 



BP P 



B P 



b._£. b... L_ b 



Encourage? 1. Smooth atrokea rather than drawing 



2. Attention to guidelines, particularly middle guidelines 



EE 



3. Careful attention to the height of the lowereaao d 
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TRY WOIID l! 

SPKLLINGl? **^»5j< 
SPELLING a? ***»<Hil< 
SPELLING 3? *** k»ock 
SPELLING 47 ••*ivo 

LOOK AT Y0U1{ SPELLINGS. 

TYPE THE NUMBER OF TIIE SPELLING YOU THINK IS XJOURECT. 
NO." THE CORRECT SPELLING IS 3 - KNOCK 

COPY THE CORRECT SPELLING FOR WORD 1 ON YOUR WORKSHEET. 
PRESS RETURN TO TRY TIIE NEXT WORD (Screen It cleared). 
TRY WORD 25 

SPELLING 1? •♦♦ililLf 
SPELLING 2? ***l\uX 
SPELLINGS? •**no 

LOOK AT YOUR SPELLINGS. 

TYPE THE NUMBER OF TIIE SPELLING YOU TIRNK IS CORRECT. 
*** I 

YES7 THE CORRECT SPELLING IS 1 ~ HALF 

COPY THE CORRECT SPELLING FOR WORD 2 ON YOUR WORKSHEET. 
(The program continues i:> his fashion for the rest of the spcUlng woi-ds on the list). 
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SPRUF Protocoli for laraple lesioni 

Introductory Menage 

IN THIS EXERCISE YOU WILL SEE A SENTENCE WITH A WORD MISSING 
AND A LIST OF WORDS THAT MIGHT BE PUT IN THE BLANK. 

TYPE THE NUMBER THAT IS IN FBOOT OF THE RIGHT WORD. 
PRESS RETURN VGR THE FIRST SENTENCE. *** 

Sample le«ton — unit words firoro nHtlllng text (Student reiponae underline^. 

SOUR IS TO .... AS LEMON IS TO SUGAR. 

X — SWET 2 ~ SWETE 3 ~ SWEET i — SWEAT 

*•« 3 

YOU* ARE RIGHT. 3 — SWEET IS CORRECT. 

PEOPLE SAY THAT A . . . . WILL EAT TIN CANS. 
I — GOTE 2 — GOAT 3 — GOUT 4 — GOAD 

I 

N0."*2 — GOAT IS CORRECT 

Sample lesson — homophones. 

. . . .MEMBERS OF THE CLUB WENT TO THE CONVENTION. 

I — ET 2 — ATE 3 — EIGHT 4 — AIT 

•*<> 

WITH HER GLASSES J^NE CAN .... CLEARLY. 

1 SE 2 •> SEA 3 — CEA 4 — SEE 

•** 

(Program continues presenting sentences until a correct response Is made for 
each sentence). 
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